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Overview 

















= Perception validation is the toughest part st re 
e Brute force road testing wont get us there ee 
e Road miles for requirements, not validation 





i 


Carnegie Mellon NREC 
Autonomous Platform 


= Multiple levels of simulation are required 
e Tie levels together via a risk reduction approach 


Demonstrator (APD) 
= Machine Learning is brittle 2008-2010 
e Low cost, synthetic fault injection can improve 
robustness 
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Validating an Autonomous Vehicle Pipeline CR ar 
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Architecture 


Perception presents a uniquely difficult assurance challenge 
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POLICE ae 





http://piximus.net/fun/funny-and-odd-things-spotted-on-the-road 
http://edtech2.boisestate.edu/robertsona/506/images/buffalo.jpg 
https://www. flickr.com/photos/hawaii-mcgraths/4458907270/in/photolist-Y59LC-7TNzZAv-5W SEds-7N24My 
https://pixabay.com/en/bunde-germany-landscape-sheep-92931/ 
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BEWARE OF 
INVISIBLE COWS 


MOST OF THE MAUNA KEA ACCESS ROAD BELOW HALE POHAKU 
IS OPEN CATTLE RANGE, AND THE COWS FREQUENTLY CROSS THE 
ROAD. DARK COLORED COWS ARE,OFTEN INVISIBLE IN DARKNESS 
AND/OR FOG. USE EXTREME CAUTION AND DRIVE VERY SLOWLY 
ll THIS OPEN RANGE. 
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Do We Need Billions of Test Miles? GQ e__, 




















= If 100M miles/critical mishap... #& WolframAlpha s==. 


e Test 3x-10x longer than mishap rate — 
=> Need 1 Billion miles of testing = 


4.03 million mi 





4.97 mi 


= Thats ~25 round trips 
on every road in the world 








e With fewer than 10 critical mishaps ~ 2 
e Then youre only as good as a human | 
— (Including the impaired humans!) a aa 


74m 360000 © 1.1 million L4 million @ ~ 2.1 million 
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Traditional Validation Meets Machine Learning ae 








= Machine Learning 
(inductive training) 
e No requirements 


— Training data is difficult to 
validate 


e No design insight 
— Generally inscrutable 
Time 
— Prone to over-fitting/gaming 






Project ) 
Definition 
Architecture 
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Verification 


_Project 
Design 


fest an 
/ Integration 
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Road “Testing Should Gather HPaUipROptS 














=m Use road miles to gather requirements 


e Novel objects, events, scenarios 
(OEDR-centric) 


e Novel operating conditions (ODD-centric) 
e Edge cases that present problems 


= Think “requirements testing’ not 
“vehicle testing’ 
e Disengagements are a blunt 
instrument for detecting novelty 


e Look for novelty even if vehicle “test” is 
passing 
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Smart Use of Simulation 
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= Point of view: everything is a simulation 
e Software & HW component simulation a 
— Includes sensor performance characterization . | 
e Software vehicle simulation 43% > \\ Cares”. 
e HIL testbeds te — at 4< 
e Closed course testing a 


— Simulated environment, obstacles, events a a 
e Public road testing 
— Assumes representativeness —_— 


s+ . 
’ 


Prd 








= Even a “perfect” simulation needs scenarios as inputs 
e You need atest plan that covers all required functionality 
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Simulation As Risk Reduction GQ *e:__., 























All Simulations Are Wrong... High Fidelity Simulation/Test; Look For Requirement Gaps 


- Environment sim. assumptions 


® Exploit the cost/fidel ity tradeoff Vehicle Testing with - Vehicle simulation artifacts Environmental simulation 


Simulated Environment ; - Vehicle simulation assumptions | quality 


B ° ti f Validation Phase Residual Risks Resolved Residual Threats to Validity 
ut some siImuU a Ions are use U Deployed Vehicle - Safety argument assumption ! Unexpected Scenarios and 
' violations detected | ODD violation modes 
] : Pre-Deployment - Scenario Exceptions | Unencountered, novel 
ea It S q\| about the assumptions Road Tests 1 - Environment Exceptions : scenarios/environment 
: ; : . - Un-modeled sensor degradation | 
@ “Perfect” simulation IS EXPpensive Closed Course Testing 1 - Environment sim. artifacts 3 drciaptelieker oltel ging 
| 


High-Resolution Vehicle & | - Simulation artifacts/assumptions ! . . 
ei Layered Strategy: Environment Simulation | - Emergent behavioral interactions ! Vehicle simulation quality 


- Vehicle dynamics ! 


e Simplifications for large search spaces Environment Simulation | ” Sensor data quality and assumptions 
@ Complex simulations for residual risks Subsystem Simulation 1 - Emergent componnent interaction | Multi-subsystem interactions 





: 2 Software Validation - Ordinary software defects | Emergent behaviors & ML 
—- Validate assumptions 


— Look for emergent effects and surprises Low Fidelity Simulation; Look for Design Defects 


= Use road tests to validate simulations 


e Identify and concentrate simulation residual risks... 


.. Which you get from analysis and road experience 
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ML Is Brittle To Adversarial Attacks [UG=a 




















Convolution Pooling Convolution Pooling Fully Fully Output Predictions 
Connected Connected 
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cat (0:04) 
boat (0.94) 
bird (0.02) 


https://goo.gl/S5sKnZV 
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Car Nota Magnified Bug Magnified Nota 
Car Difference Difference Bus 


Szegedy, Christian, et al. "Intriguing properties of neural networks.” arXiv preprint arXiv:1312.6199 (2013). © 2018 Edge Case ResearchLLC 10 
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Sensor data corruption Experiments at NREC 


Gaussian blur 





: oa : 
ae: 


(Golic-lame(-\(-aile)p False negative 


Exploring the response of a DNN to environmental 
perturbations from “Robustness Testing for 
Perception Systems,” RIOT Project, NREC, DIST-A. 
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Sensor Fault Injection 

















a Synthesize faults during development ve << 
e Degrade sensor inputs at 

— Use simple approximate models 
e Insert slightly malformed objects 


e Even ‘unrealistic’ faults provide insight 
i. ae a Missed: 














= Realistic data for validation 
e Road data when you have it 
e Highly realistic synthetic faults 
— E.g., photo-realistic haze 


ee ~Noise + 
PL Black Car 


Pedestrian 
Missed: 
Gaussian 
Blur 
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Multiple Sensors _ O's... 


a bay injection needs to coordinate multiple sensor inputs 





© 2018 Edge Case Research LLC 13 











Ideas for Better AV Sensor Validation Qe... 





1. Concentrate on data collection with road miles 
e Look for things beyond disengagement triggers | 
e Road tests should validate, not debug 


2. Use a layered approach to simulation 
e Exploit fidelity/cost tradeoffs 
e Validate assumptions & simplifications 





3. Sensor-level fault injection can help 
e Generative models to create test scenarios 
e Fault injection to improve robustness 
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